We're essentially.

At a place.

We've been before if you remember back we started this

master with loads of probability and modeling stuff.

And all of those.

We're really only.

So that the agent has an idea of what the world is.

So that actually can choose an action.

We've done that in static worlds.

First.

And now we've done the probability theory modeling thing

a magic.

For dynamic worlds.

So I can.

Imagine.

What the next thing is we're going to make decision.

In dynamic world.

And.

Essentially, what we're going to look at is something called

Markov decision.

Procedures, which is actually how to make decisions in

Markovian worlds.

Using Markov models.

We're going to look at.

Some algorithms for that.

And.

Then.

We're looking at partially observable versions of these

Markov decision.

Procedures.

Then we're going to finally look at agents for this kind of

most general.

Were general.

Situation in which we can have agents.

The idea is that.

We can now do decisions in sequential worlds.

You remember.

We could do decision in episodic worlds.

Where essentially we have rounds based.

Structure where nothing happens while you think and you make a

decision and then a new world starts essentially.

And now we involve more and more time.

And that gives us sequential worlds and.

The main limitation we still have is that everything is

discrete.

And.

Which is something you can lift if you have more mathematics

than I was willing to introduce here.

Which basically means wherever you see a some you have to do an

integral.